11 research outputs found
CATS: linearizability and partition tolerance in scalable and self-organizing key-value stores
Distributed key-value stores provide scalable, fault-tolerant, and self-organizing
storage services, but fall short of guaranteeing linearizable consistency
in partially synchronous, lossy, partitionable, and dynamic networks, when data
is distributed and replicated automatically by the principle of consistent hashing.
This paper introduces consistent quorums as a solution for achieving atomic
consistency. We present the design and implementation of CATS, a distributed
key-value store which uses consistent quorums to guarantee linearizability and partition tolerance in such adverse and dynamic network conditions. CATS is
scalable, elastic, and self-organizing; key properties for modern cloud storage
middleware. Our system shows that consistency can be achieved with practical
performance and modest throughput overhead (5%) for read-intensive workloads
Handling Network Partitions and Mergers in Structured Overlay Networks
Structured overlay networks form a major class of peer-to-peer systems, which are touted for their abilities to
scale, tolerate failures, and self-manage. Any long-lived
Internet-scale distributed system is destined to face network partitions. Although the problem of network partitions
and mergers is highly related to fault-tolerance and
self-management in large-scale systems, it has hardly been
studied in the context of structured peer-to-peer systems.
These systems have mainly been studied under churn (frequent
joins/failures), which as a side effect solves the problem
of network partitions, as it is similar to massive node
failures. Yet, the crucial aspect of network mergers has been
ignored. In fact, it has been claimed that ring-based structured
overlay networks, which constitute the majority of the
structured overlays, are intrinsically ill-suited for merging
rings. In this paper, we present an algorithm for merging
multiple similar ring-based overlays when the underlying
network merges. We examine the solution in dynamic conditions,
showing how our solution is resilient to churn during
the merger, something widely believed to be difficult or
impossible. We evaluate the algorithm for various scenarios
and show that even when falsely detecting a merger, the
algorithm quickly terminates and does not clutter the network
with many messages. The algorithm is flexible as the
tradeoff between message complexity and time complexity
can be adjusted by a parameter
Partition Tolerance and Data Consistency in Structured Overlay Networks
Structured overlay networks forma major class of peer-to-peer systems, which are used to build scalable, fault-tolerant and self-managing distributed applications. This thesis presents algorithms for structured overlay networks, on the routing and data level, in the presence of network and node dynamism.
On the routing level, we provide algorithms for maintaining the structure of the overlay, and handling extreme churn scenarios such as bootstrapping, and network partitions and mergers. Since any long lived Internet-scale distributed system is destined to face network partitions, we believe structured overlays should intrinsically be able to handle partitions and mergers. In this thesis, we discuss mechanisms for detecting a network partition and merger, and provide algorithms for merging multiple ring-based overlays. Next, we present a decentralized algorithm for estimating the number of nodes in a peer-to-peer system. Lastly, we discuss the causes of routing anomalies (lookup inconsistencies), their effect on data consistency, and mechanisms on the routing level to reduce data inconsistency.
On the data level, we provide algorithms for achieving strong consistency and partition tolerance in structured overlays. Based on our solutions on the routing and data level, we build a distributed key-value store for dynamic partially synchronous networks, which is linearizable, self-managing, elastic, and exhibits unlimited linear scalability. Finally,we present a replication scheme for structured overlays that is less sensitive to churn than existing schemes, and allows different replication degrees for different key ranges that enables using higher number of replicas for hotspots and critical data
Handling Network Partitions and Mergers in Structured Overlay Networks
Structured overlay networks form a major class of peer-to-peer systems, which are touted for their abilities to scale, tolerate failures, and self-manage. Any long-lived Internet-scale distributed system is destined to face network partitions. Although the problem of network partitions and mergers is highly related to fault-tolerance and self-management in large-scale systems, it has hardly been studied in the context of structured peer-to-peer systems. These systems have mainly been studied under churn (frequent joins/failures), which as a side effect solves the problem of network partitions, as it is similar to massive node failures. Yet, the crucial aspect of network mergers has been ignored. In fact, it has been claimed that ring-based structured overlay networks, which constitute the majority of the structured overlays, are intrinsically ill-suited for merging rings. In this paper, we present an algorithm for merging multiple similar ring-based overlays when the underlying network merges. We examine the solution in dynamic conditions, showing how our solution is resilient to churn during the merger, something widely believed to be difficult or impossible. We evaluate the algorithm for various scenarios and show that even when falsely detecting a merger, the algorithm quickly terminates and does not clutter the network with many messages. The algorithm is flexible as the tradeoff between message complexity and time complexity can be adjusted by a parameter